Imaging apparatus, image processing method, and program

ABSTRACT

A defective pixel of an imaging device is detected. An output pixel value of the defective pixel is corrected to generate an output image. An imaging device, and a signal processing unit for analyzing an output signal from the imaging device and detecting a defective pixel are included. The imaging device receives incident light via, for example, a microlens placed in front of a pixel, inputs the same subject light on a local area basis including a plurality of pixels of the imaging device, and acquires an image signal lower than a pixel resolving power corresponding to the pixel density of the imaging device. The signal processing unit compares the pixel values of the same color pixels included in a local area on a local area basis including a cluster of the plurality of pixels of the imaging device, detects the defective pixel based on the comparison result, and corrects and outputs a pixel value of a pixel determined to be a defective pixel.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of and claims the benefit under 35U.S.C. §120 of U.S. patent application Ser. No. 14/435,806, titled“IMAGING APPARATUS, IMAGE PROCESSING METHOD, AND PROGRAM,” filed on Apr.15, 2015, which is a National Stage of International Application No.PCT/JP2013/075639, filed in the Japanese Patent Office as a ReceivingOffice on Sep. 24, 2013, which claims priority to Japanese PatentApplication Number JP 2012-234004, filed in the Japanese Patent Officeon Oct. 23, 2012, each of which is hereby incorporated by reference inits entirety.

TECHNICAL FIELD

The present disclosure relates to an imaging apparatus, an imageprocessing method, and a program, and especially relates to an imagingapparatus, image processing method, and program that detect a defectivepixel of an imaging device.

BACKGROUND ART

An imaging device mounted in a digital camera, video camera, or the likeincludes a CMOS (Complementary Metal Oxide Semiconductor) image sensor,or a CCD (Charge Coupled Device) image sensor, which is formed on asemiconductor substrate. A defective pixel may occur in such an imagingdevice due to a local crystal defect on the semiconductor substrate.

Such a defective pixel ends up outputting a specific pixel value that isnot dependent on the amount of incident light. Therefore, if thedefective pixel exists in the imaging device, a captured image includingan error pixel value is output so that the image quality deteriorates.

Many of the known imaging apparatuses such as video cameras and digitalcameras have the following countermeasure configuration to prevent anerror based on such a defective pixel. In other words, address dataindicating the location of a defective pixel in a solid-state imagingdevice is prestored in a memory device such as nonvolatile memory in themanufacturing stage. A signal output from the defective pixel iscorrected and output based on the address data indicating the defectivepixel stored in the memory device upon image capture. This is thecountermeasure.

For example, in a case where a CMOS image sensor is mounted in animaging apparatus, an output signal of each pixel of the CMOS imagesensor and output signals of neighboring pixels of the pixel to beexamined are compared in the manufacturing stage in a state where a lensof the imaging apparatus is shielded from light and light does not enterthe CMOS image sensor. If the difference between these output signalvalues exceeds a predetermined threshold value, the pixel that hasoutput this specific signal is determined to be a defective pixel.

Furthermore, address data indicating the location of the pixeldetermined to be a defective pixel is stored in nonvolatile memory. Whenthe defective pixel detection ends, the CMOS image sensor and thenonvolatile memory in which the address data of the defective pixel isstored are integrated in the imaging apparatus for shipment.

When a user captures an image using the imaging apparatus, an outputsignal of the defective pixel included in an output video signal fromthe CMOS image sensor is corrected, using output signals of the pixelsin the vicinity of the detective pixel, based on the address data of thedefective pixel of the CMOS image sensor stored in the nonvolatilememory. An image having the corrected pixel value signal is then outputas an output image.

Such a method is used as the known countermeasure against defectivepixels in many cases.

However, the process is effective strictly and only for a defectivepixel found in the manufacturing stage of the imaging device. There is aproblem that the above countermeasure cannot deal with a defective pixelof the imaging device occurring due to aging after the shipment of theimaging apparatus.

In order to deal with such a problem, for example, Patent Document 1 (JP06-6685 A) discloses a defect correction apparatus that, upon theturning-on of the power to an imaging apparatus, closes an aperture of alens mounted in the imaging apparatus to enter a light-shielding state,detects a defective pixel with an imaging output signal of a solid-stateimaging device, records and holds defect data based on a detectionsignal from the defective pixel, and corrects the defective pixel, usingthe latest defect data upon image capture.

However, such a defect correction configuration needs to store addressdata of the defective pixel, and the like in memory, and memory capacitytherefor needs to be secured. Accordingly, there is a problem to resultin an increase in the cost of the apparatus. Moreover, there is also aproblem that the number of correctable defective pixels tends to dependon the storage capacity of the memory in which the address data of thedefective pixels is stored.

Furthermore, for example, Patent Document 2 (JP 2008-154276 A) andPatent Document 3 (JP 2009-105872 A) disclose configurations that alsoenable the detection and correction of a defective pixel that occursafter shipment without mounting a memory in which the locations ofdefective pixels are stored.

Patent Document 2 discloses the configuration that verifies, forexample, a texture direction around a pixel of interest selected from acaptured image and detects and corrects a defect with reference to thetexture direction. The detection accuracy and the correction accuracyare improved by the process with reference to the texture direction.

However, there is a problem with the configuration that the performancein the detection and correction of a defect is decreased in an imagearea where the texture direction around a pixel of interest cannot beobtained accurately, such as a flat area. Moreover, there is also aproblem that the calculation cost and the circuit size are increased todetermine the texture direction accurately.

Moreover, Patent Document 3 discloses the configuration that uses thestandard deviation value of neighboring pixels for the detection of adefective pixel based on a captured image. However, the value of thestandard deviation is also increased at an edge of the image.Accordingly, there is a problem that such a detection error as thatdetermines the edge area to be a defective pixel occurs, and a wrongpixel value correction is made to contrarily deteriorate the imagequality.

CITATION LIST Patent Document

-   Patent Document 1: JP 06-6685 A-   Patent Document 2: JP 2008-154276 A-   Patent Document 3: JP 2009-105872 A

SUMMARY OF THE INVENTION Problems to be Solved by the Invention

The present disclosure has been made considering, for example, the aboveproblems, and an object thereof is to provide an imaging apparatus,image processing method, and program that highly accurately detect adefective pixel existing in an imaging device, and generate an outputimage of high quality where the output pixel value of the defectivepixel has been corrected.

Solutions to Problems

A first aspect of the present disclosure is an imaging apparatusincluding:

an imaging device; and

a signal processing unit for analyzing an output signal from the imagingdevice and detecting a defective pixel included in the imaging device,wherein

the imaging device inputs the same subject light on a local area basisincluding a plurality of pixels of the imaging device, acquires an imagesignal lower than a pixel resolving power corresponding to the pixeldensity of the imaging device, and outputs the image signal to thesignal processing unit, and

the signal processing unit compares pixel values of the same colorpixels included in the local area on a local area basis including acluster of the plurality of pixels of the imaging device, and detects adefective pixel based on the comparison result.

Further, in an embodiment of the imaging apparatus of the presentdisclosure, the signal processing unit executes a process of determininga pixel of interest to be a defective pixel when variations in pixelvalues of the same color pixels included in the local area is large, andwhen a difference between an average of intermediate pixel values exceptmaximum and minimum pixel values of the same color pixels included inthe local area, and a pixel value of the pixel of interest is large.

Further, in an embodiment of the imaging apparatus of the presentdisclosure, the signal processing unit executes a process of determininga pixel of interest to be a defective pixel when a standard deviation ofpixel values of the same color pixels included in the local area islarger than a preset threshold value (TH1), and when a differenceabsolute value between an average of intermediate pixel values exceptmaximum and minimum pixel values of the same color pixels included inthe local area, and a pixel value of the pixel of interest is largerthan a preset threshold value (TH2).

Further, in an embodiment of the imaging apparatus of the presentdisclosure, the imaging device includes a photoelectric conversion unithaving pixels arranged in a two-dimensional array form, and a microlensplaced on an imaging lens side being on a front side of thephotoelectric conversion unit, and has a configuration where incidentlight via an imaging lens is diffused via the microlens, and the samesubject light is received on the local area basis including theplurality of pixels of the imaging device.

Further, in an embodiment of the imaging apparatus of the presentdisclosure, the signal processing unit has a configuration where animage on a specific pixel area basis is reconstructed from an imageacquired by the imaging device including the microlens and accordingly aspecific point-of-view image is generated.

Further, in an embodiment of the imaging apparatus of the presentdisclosure, the imaging device has a pixel arrangement where each pixelof a Bayer arrangement including RGB pixels or an arrangement includingRGBW pixels is split into four, 2×2 pixels, of the same color.

Further, in an embodiment of the imaging apparatus of the presentdisclosure, the imaging device inputs the same subject light on a localarea basis including 2×2 same color pixels of the imaging device,acquires an image signal lower than a pixel resolving powercorresponding to the pixel density of the imaging device, and outputsthe image signal to the signal processing unit.

Further, in an embodiment of the imaging apparatus of the presentdisclosure, the imaging device inputs the same subject light on a localarea basis including 4×4 or 8×8 pixels of the imaging device, acquiresan image signal lower than a pixel resolving power corresponding to thepixel density of the imaging device, and outputs the image signal to thesignal processing unit.

Further, in an embodiment of the imaging apparatus of the presentdisclosure, the imaging device lets in subject light via alow-resolution imaging lens with a low optical resolving power forforming an optical image with a resolving power lower than a pixelresolving power corresponding to the pixel density of the imaging deviceand accordingly acquires an image signal lower than the pixel resolvingpower corresponding to the pixel density of the imaging device, andoutputs the image signal to the signal processing unit.

Further, in an embodiment of the imaging apparatus of the presentdisclosure, the imaging device includes a photoelectric conversion unithaving pixels arranged in a two-dimensional array form, and an opticallow-pass filter placed on an imaging lens side being on a front side ofthe photoelectric conversion unit, and has a configuration whereincident light via an imaging lens is diffused via the optical low-passfilter, and the same subject light is received on the local area basisincluding the plurality of pixels of the imaging device.

Further, in an embodiment of the imaging apparatus of the presentdisclosure, the signal processing unit includes a defective pixelcorrection unit for executing a pixel value correction on a defectivepixel, and the defective pixel correction unit calculates a correctedpixel value of the defective pixel taking, as reference pixels,intermediate pixel values except maximum and minimum pixel values of thepixels of the same color as the defective pixel included in the samelocal area as the defective pixel.

Further, in an embodiment of the imaging apparatus of the presentdisclosure, the defective pixel correction unit sets a maximum pixelvalue of the intermediate pixel values as the corrected pixel value ofthe defective pixel when the defective pixel has the maximum pixel valueamong the same color pixels in the local area including the defectivepixel, and sets a minimum pixel value of the intermediate pixel valuesas the corrected pixel value of the defective pixel when the defectivepixel has the minimum pixel value among the same color pixels in thelocal area including the defective pixel.

Further, a second aspect of the present disclosure is an imageprocessing method executed in an imaging apparatus, executing:

a step of, in an imaging device, inputting the same subject light on alocal area basis including a plurality of pixels of the imaging device,acquiring an image signal lower than a pixel resolving powercorresponding to the pixel density of the imaging device, and outputtingthe image signal to the signal processing unit, and

a signal processing step of, in the signal processing unit, analyzing anoutput signal from the imaging device, and detecting a defective pixelincluded in the imaging device, wherein

in the signal processing step, the signal processing unit compares pixelvalues of the same color pixels included in the local area on a localarea basis including a cluster of the plurality of pixels of the imagingdevice, and detects the defective pixel based on the comparison result.

Further, a third aspect of the present disclosure is a program to causean imaging apparatus to execute image processing, including:

a step of causing an imaging device to input the same subject light on alocal area basis including a plurality of pixels of the imaging device,acquire an image signal lower than a pixel resolving power correspondingto the pixel density of the imaging device, and output the image signalto the signal processing unit, and

a signal processing step of causing the signal processing unit toanalyze an output signal from the imaging device, and detect a defectivepixel included in the imaging device, wherein

the signal processing step includes comparing pixel values of the samecolor pixels included in the local area on a local area basis includinga cluster of the plurality of pixels of the imaging device, anddetecting the defective pixel based on the comparison result.

The program of the present disclosure is, for example, a program thatcan be provided by a storage medium or communication medium that isprovided in a computer readable form, to an information processingapparatus or computer system that can execute various program codes.Such a program is provided in the computer readable form. Accordingly,processes in accordance with the program can be achieved on theinformation processing apparatus or computer system.

Still other objects, features, and advantages of the present disclosurewill be clear from a more detailed description based on examples of thepresent disclosure described below and the accompanying drawings. Thesystem in the present description is a logically assembled configurationof a plurality of devices, and is not limited to one having devices ofconfigurations in the same housing.

Effects of the Invention

According to a configuration of one example of the present disclosure, adefective pixel of an imaging device is detected. An output pixel valueof the defective pixel is corrected to generate an output image.

Specifically, an imaging apparatus includes an imaging device, and asignal processing unit that analyzes an output signal from the imagingdevice and detects a defective pixel. The imaging device receivesincident light via, for example, a microlens placed in front of a pixel,inputs the same subject light on a local area basis including aplurality of pixels of the imaging device, and acquires an image signallower than a pixel resolving power corresponding to the pixel density ofthe imaging device. The signal processing unit compares the pixel valuesof the same color pixels included in the local area on a local areabasis including a cluster of the plurality of pixels of the imagingdevice, detects a defective pixel based on the comparison result, andcorrects and outputs a pixel value of the pixel determined to be adefective pixel.

These processes enable highly accurate detection of a defective pixelexisting in the imaging device. Accordingly, it is possible to generatea high quality output image where an output pixel value of the defectivepixel has been corrected.

BRIEF DESCRIPTION OF DRAWINGS

FIGS. 1(1) and 1(2) are diagrams explaining pixel configurations of aBayer arrangement and a four-way split Bayer arrangement.

FIGS. 2(a) and 2(b) are diagrams explaining a general imagingconfiguration and a configuration of an imaging device of an imagingapparatus of the present disclosure.

FIGS. 3(1), 3(2 a), and 3(2 b) are diagrams explaining a captured imageof the imaging apparatus of the present disclosure.

FIG. 4 is a diagram explaining the configuration of the imaging deviceof the imaging apparatus of the present disclosure.

FIGS. 5(b) and 5(c) are diagrams explaining a captured image of theimaging apparatus of the present disclosure.

FIG. 6 is a diagram explaining an example of the entire configuration ofthe imaging apparatus of the present disclosure.

FIG. 7 is a diagram explaining one example of a signal processing unitof the imaging apparatus of the present disclosure.

FIG. 8 is a diagram explaining an example of the configuration of adefect detection and correction unit of the signal processing unit ofthe imaging apparatus of the present disclosure.

FIG. 9 is a diagram explaining an example of the detailed configurationof a defective pixel detection unit of the defect detection andcorrection unit.

FIG. 10 is a diagram illustrating a flowchart explaining a processsequence to be executed by the defective pixel detection unit of thedefect detection and correction unit.

FIG. 11 is a diagram explaining an example of the detailed configurationof a defective pixel correction unit of the defect detection andcorrection unit.

FIG. 12 is a diagram illustrating a flowchart explaining a processsequence to be executed by the defective pixel correction unit of thedefect detection and correction unit.

FIG. 13 is a diagram explaining an example of the configuration of theimaging device of the imaging apparatus of the present disclosure.

FIG. 14 is a diagram explaining an example of the configuration of theimaging device of the imaging apparatus of the present disclosure.

FIGS. 15(A) and 15(B) are diagrams explaining examples of the pixelarrangement of the imaging device of the imaging apparatus of thepresent disclosure.

FIG. 16 is a diagram explaining one example of the signal processingunit of the imaging apparatus of the present disclosure.

FIG. 17 is a diagram explaining one example of the signal processingunit of the imaging apparatus of the present disclosure.

FIG. 18 is a diagram explaining a configuration that individuallyacquires an image from a different point-of-view, or an image with adifferent focal length, depending on the area of a photoelectricconversion element of the imaging device.

MODE FOR CARRYING OUT THE INVENTION

Hereinafter, the details of an imaging apparatus, an image processingmethod, and a program of the present disclosure are described withreference to the drawings. The descriptions are given in accordance withthe following items:

1. Correspondence between an Imaging Device Configuration and IncidentLight in the Imaging Apparatus of the Present Disclosure;

2. Configuration and Process of the Imaging Apparatus of the PresentDisclosure;

3. Configuration and Process of the Defect Detection and CorrectionUnit;

3-1. Details of the Configuration and Process of the Defective PixelDetection Unit;

3-2. Details of the Configuration and Process of the Defective PixelCorrection Unit;

4. Example Using a Low-resolution Lens for the Imaging Lens;

5. Example Using an Optical Low-pass Filter;

6. Variations in the Pixel Arrangement;

7. Example in which the Arrangement Conversion Unit Is Replaced with theDemosaicing Processing Unit,

8. Example Including a Specific Point-of-View Image Generation Unit; and

9. Summary of the Configuration of the Present Disclosure.

[1. Correspondence Between an Imaging Device Configuration and IncidentLight in the Imaging Apparatus of the Present Disclosure]

Firstly, a description is given of the correspondence between an imagingdevice configuration and incident light in the imaging apparatus of thepresent disclosure. FIGS. 1(1) and 1(2) illustrate the following twoexamples of the pixel arrangement of an imaging device (image sensor).

(1) A Bayer RGB Arrangement

(2) A Four-way split Bayer RGB arrangement

The Bayer RGB arrangement illustrated in FIG. 1(1) is the pixelarrangement of an imaging device (image sensor) used in current digitalcameras, video cameras, and the like in many cases, and has aconfiguration where each pixel selectively receives one of R (red), G(green), and B (blue) lights. Specifically, a color filter thatselectively transmits one of the RGB wavelength lights is attached infront of a photoelectric conversion element to selectively receive oneof the RGB wavelength lights in each pixel of the imaging device.

Each of the RGB pixels stores charge in accordance with the lightreceiving amount, on a pixel basis. Each pixel outputs an electricalsignal, in other words, a pixel value in accordance with the storedcharge.

On the other hand, the four-way split Bayer RGB arrangement illustratedin FIG. 1(2) is an example of the configuration of an imaging device(image sensor) applied in the imaging apparatus (the imaging apparatus)of the present disclosure. The four-way split Bayer RGB arrangementillustrated in FIG. 1(2) includes a pixel arrangement where each of theRGB pixels of the Bayer RGB arrangement illustrated in FIG. 1(1) issplit into four.

In other words, one R pixel of the Bayer RGB arrangement illustrated inFIG. 1(1) is split into four to make it four R pixels. One G pixel issplit into four to make it four G pixels. One B pixel is split into fourto make it four B pixels.

As the entire imaging device, the number of pixels of the four-way splitBayer RGB arrangement illustrated in (2) has the number of pixels fourtimes as many as the number of pixels of the Bayer RGB arrangementillustrated in (1).

In recent years, the pixel density of the imaging device can bedramatically increased with the progress of the semiconductortechnology. The imaging device having the dramatically increased numberof pixels can be created. Accordingly, such an imaging device with ahigh pixel density as illustrated in FIG. 1(2) can be manufactured.

The imaging device of the four-way split Bayer RGB arrangementillustrated in FIG. 1(2) has a four-times resolving power as compared tothe imaging device of the Bayer RGB arrangement illustrated in FIG.1(1).

However, in the imaging apparatus of the present disclosure, the imagingdevice of the four-way split Bayer RGB arrangement illustrated in FIG.1(2) is applied. An image is captured which has a resolving power thathas been decreased on purpose to a resolving power level of the imagingdevice of the Bayer RGB arrangement illustrated in FIG. 1(1). The highlyaccurate detection of a defective pixel is achieved using the capturedimage with the decreased resolving power level.

An example of an imaging configuration in the imaging apparatus of thepresent disclosure is described with reference to FIGS. 2(a) and 2(b).

FIGS. 2(a) and 2(b) illustrate the following two imaging configurations:

(a) A general imaging configuration; and

(b) One example of an imaging configuration of the imaging apparatus ofthe present disclosure.

If image capture making use of the resolving power of the imaging deviceis performed, an image is generally captured by performing focus controlon incident light via an imaging lens 110 in such a manner as to form asubject image being an image capture target on each pixel of an imagingdevice 120, as illustrated in FIG. 2(a).

Each pixel of the imaging device has a configuration where aphotoelectric conversion element (photodiode) 121, a color filter 122,and an onchip lens 123 are stacked. Each photoelectric conversionelement 121 stores charge in accordance with the intensity of awavelength light transmitted through the color filter 122 in front ofthe photoelectric conversion element 121, in other words, a wavelengthlight corresponding to one of R, G, and B, and outputs a signal (pixelvalue) proportional to the stored charge.

The imaging process illustrated in FIG. 2(a) enables the imaging of animage of a different subject area depending on the pixel of the imagingdevice. Accordingly, an image with a resolving power in accordance withthe pixel density of the imaging device can be captured.

In contrast, in the imaging configuration according to one example ofthe present disclosure illustrated in FIG. 2(b), the imaging device 120has a different configuration from that of the imaging deviceillustrated in FIG. 2(a). In other words, the microlens 124 is includedon the front side of, in other words, on the imaging lens 110 side, ofthe imaging device 120.

The microlens 124 performs the process of diffusing the incident lightentering via the imaging lens 110. With the diffusion process, an imagelight that should be condensed onto one pixel illustrated in FIG. 2(a)is diffused over and enters a pixel area of a plurality of pixels, forexample, 2×2=4 pixels, of the photoelectric conversion elements 121.

With such an imaging process, a captured image acquired by the imagingdevice 120 becomes an image with low resolution that does not haveresolution corresponding to the resolving power of the imaging device.However, it becomes possible to detect a defective pixel with highaccuracy. The process is described later.

A more specific captured image of the imaging configuration illustratedin FIG. 2(b) is described with reference to FIGS. 3(1), 3(2 a), and 3(2b). FIGS. 3(1), 3(2 a), and 3(2 b) are diagrams schematicallyillustrating imaging configurations for the two imaging devicearrangements described above with reference to FIGS. 1(1) and 1(2).FIGS. 3(1), 3(2 a), and 3(2 b) illustrate the following imagingconfiguration examples:

(1) An imaging configuration for the known Bayer arrangement;

(2) As examples of the imaging configuration of the four-way split Bayerarrangement;

(2 a) An imaging configuration with a setting to form a subject image oneach pixel, illustrated in FIGS. 2(a); and

(2 b) An imaging configuration with a setting to form a subject image ona plurality of pixels (2×2 pixels), illustrated in FIG. 2(b)

The imaging configuration for the known Bayer arrangement illustrated inFIG. 3(1) forms images of individually different areas of the subjectimage on the pixels, that is, the RGB pixels, of the imaging device. RGBillustrated in the figures illustrate the RGB pixels of the known Bayerarrangement. Here, only 2×2=4 pixels are illustrated. Each coneindicated by dotted lines conceptually illustrates incident lightcorresponding to a different area of the subject.

With the configuration, for example, an image with resolution inaccordance with the number of pixels of the imaging device can beobtained.

FIGS. 3(2 a) and (2 b) illustrate the following two examples as examplesof the imaging configuration for the four-way split Bayer arrangementillustrated in FIG. 1(2):

FIG. 3(2 a) is an imaging configuration that is illustrated in FIG. 2(a)and has a setting where a subject image is formed on each pixel.

The imaging configuration illustrated in FIG. 3(2 a) is a configurationto form images of individually different areas of the subject image onthe pixels of the imaging device as in FIG. 3(1). If such an imagingprocess is performed, an image can be obtained which has four-timesresolution as compared to the captured image of the imaging device ofthe known Bayer arrangement of FIG. 3(1).

In contrast, FIG. 3(2 b) is another example of the imaging configurationof the four-way split Bayer arrangement illustrated in FIG. 1(2), andcorresponds to the imaging configuration of FIG. 2(b). In other words,it is an imaging configuration having a setting where a subject image isformed not on a single pixel but on a plurality of pixels (2×2=4pixels). Compared to FIG. 3(2 a), an image has reduced resolution. Inother words, the resolution level is similar to the level of the imageobtained by the imaging process using the known Bayer arrangementillustrated in FIG. 3(1).

In this manner, the imaging apparatus of the present disclosurecondenses incident light with more coarse resolution than the resolvingpower of the imaging device (image sensor). In other words, an image iscaptured which has an optical resolving power lower than a pixelresolving power specified by the pixel density of the imaging device.

In many cases, an imaging apparatus using an imaging device withhigh-density pixels includes an imaging unit by applying an opticalelement, such as a lens, having an optical resolving power in accordancewith a pixel resolving power corresponding to the pixel density of theimaging device. Such agreement between the pixel resolving power and theoptical resolving power makes it possible to form a subject image thatis different depending on the pixel, as described above with referenceto FIG. 2(a). Accordingly, an image can be captured which has resolutionin accordance with the pixel density of the imaging device.

However, the imaging apparatus of the present disclosure captures animage with lower resolution than the pixel resolving power of theimaging device on purpose, as described with reference to FIGS. 2(b) and3(2 b). In other words, it is a configuration where the opticalresolving power has been reduced as compared to the pixel resolvingpower.

Various configurations can be used as the configuration for acquiring animage with the optical resolving power lower than the pixel resolvingpower. In a first example described below uses the microlenses 124described with reference to FIG. 2(b).

The configuration of the imaging device including the microlenses 124 isfurther described with reference to FIG. 4.

FIG. 4 illustrates a cross-sectional configuration of the imaging lens110 and the imaging device 120. It is a similar diagram to the onedescribed above with reference to FIG. 2(b). Furthermore, FIG. 4(a)illustrates a schematic diagram of an imaging configuration for 2×2=4pixels for a similar imaging device to the one of FIG. 3(2 b).

As illustrated in FIG. 4, the imaging device 120 has a configurationwhere the color filters 122 in which a color (RGB) corresponding to eachpixel is set are placed on the photoelectric conversion elements(photodiodes) 121 including pixels arranged in a two-dimensionally arrayform, and the onchip lenses 123 are further mounted thereon.

The imaging device 120 of the example has a configuration where themicrolenses 124 are further placed on the onchip lenses 123 asillustrated in FIG. 4.

As described above, light emitted from the subject in focus is generallydesigned to be condensed at a position of each photoelectric conversionelement (photodiode) via the imaging lens 110. In other words, it is theimaging configuration described above with reference to FIG. 2(a).

However, the imaging apparatus of the present disclosure condenses thelight emitted from the subject onto the microlens 124 via the imaginglens 110, and applies the light spreading over 2N×2N (N is any integer)pixels (photoelectric conversion elements).

The example illustrated in FIG. 4(a) is a setting to diffuse incidentlight from the same subject area over 2×2 pixels in a case of N=1, andlet in the light, as in the above description with reference to FIG. 3(2b).

With such a configuration, light of substantially the same amount enterseach pixel included in a cluster of pixels, in other words, eachindividual pixel forming 2×2=4 pixels, in a local area exposed to thediffused light.

Therefore, each output of the 2×2=4 pixels should be a uniform outputvalue (pixel value).

However, if a defective pixel is included in one pixel cluster where thesame subject light is diffused and enters, in other words, 2×2=4 pixelsillustrated in FIG. 4(a) in the example, the output value (pixel value)of only the defective pixel has a different value from the output values(pixel values) of the other pixels (normal pixels) among the pixelcluster (four pixels). In this manner, if the output pixel values ofindividual pixels of one pixel cluster where the same subject lightenters are not uniform, the pixel cluster can be estimated to include adefective pixel.

In this manner, the imaging apparatus of the present disclosure comparesthe output values (pixel values) of pixels forming a pixel cluster on apixel cluster basis where the same subject light enters, determines thatthe pixel cluster include a defective pixel if the outputs are notuniform, and further performs the process of identifying the defectivepixel.

With such a process, it becomes possible to detect a defective pixelwith high accuracy without confusing, for example, an edge area of animage with a defective pixel.

The output pixel value of the detected defective pixel is corrected in asignal processing unit of the imaging apparatus, and then output.

In the example, the color filter arrangement is described taking anexample where the four-way split Bayer RGB arrangement described withreference to FIG. 1(2) is used. However, the processing of the presentdisclosure can be applied to imaging devices having various arrangementsother than the four-way split Bayer RGB arrangement.

Moreover, FIGS. 3 (2 b) and 4 (a) illustrate the configuration of onepixel cluster where the same subject light is diffused and enters,taking an example of 2×2=4 pixels. However, the setting of the pixelcluster is not limited to such a setting of 2×2 pixels and varioussettings are possible.

Various settings, for example, a setting, illustrated in FIG. 5(b), inwhich the pixel cluster where the same subject light is diffused andenters is 4×4=16 pixels and, a setting, illustrated in FIG. 5(c), inwhich the pixel cluster where the same subject light is diffused andenters is 8×8=64 pixels are possible.

However, upon the comparison of pixel values in a case of detecting adefective pixel, the output values of the same color pixels in a pixelcluster where the same subject light is diffused and enters arecompared.

[2. Configuration and Process of the Imaging Apparatus of the PresentDisclosure]

Next, configuration and process examples of the imaging apparatus of thefirst example of the present disclosure are described with reference toFIG. 6 and later figures.

FIG. 6 is a diagram illustrating the entire configuration of an imagingapparatus 100 according to the first example of the present invention.

The light incident via the optical lens 110 enters the imaging unit, forexample, the imaging device 120 including a CMOS image sensor or thelike to output image data by photoelectric conversion.

The imaging device 120 is the imaging device 120 having, for example,the configuration illustrated in FIG. 4. For example, the imaging device120 has the four-way split Bayer RGB arrangement described withreference to FIG. 1(2), and has clusters of pixels, in each of which thesame subject light is diffused and enters, as illustrated in FIG. 4(a),FIGS. 5(b), 5(c), or the like.

The image data output from the imaging device 120 illustrated in FIG. 6is input into a signal processing unit 130. The signal processing unit130 executes processes, for example, the detection of a defective pixeland the correction of the pixel value of the defective pixel, which aredescribed below, further executes general signal processing in a camera,such as white balance (WB) adjustment and gamma correction, andgenerates an output image 300. The output image 300 is stored in anunillustrated storage unit, or output to a display unit.

A control unit 135 outputs a control signal to each unit in accordancewith a program stored in, for example, the unillustrated memory, andperforms the control of various processes.

Next, an example of the configuration of the signal processing unit 130is described with reference to FIG. 7.

FIG. 7 is a diagram illustrating the configuration of the signalprocessing unit 130 of one example of the present disclosure.

The signal processing unit 130 includes a defect detection andcorrection unit 140, an arrangement conversion unit 170, and a camerasignal processing unit 180, as illustrated in FIG. 7.

The imaging device 120 has the pixel arrangement described withreference to FIG. 1(2), and generates and outputs a four-way split Bayerarrangement image 211.

The four-way split Bayer arrangement image 211 output by the imagingdevice 120 includes clusters of pixels, in each of which the samesubject light is diffused and enters, as illustrated with reference toFIG. 4.

The four-way split Bayer arrangement image 211 output from the imagingdevice 120 is input into the defect detection and correction unit 140.

The defect detection and correction unit 140 inputs the four-way splitBayer arrangement image 211 from the imaging device 120, analyzes thepixel values of pixels forming the image, and detects a defective pixel.Furthermore, the process of correcting the pixel value of the detecteddefective pixel is executed. As a result, a defect-corrected four-waysplit Bayer arrangement image 212 is generated and output.

The defect-corrected four-way split Bayer arrangement image 212generated by the defect detection and correction unit 140 is input intothe arrangement conversion unit 170.

The arrangement conversion unit 170 inputs the defect-corrected four-waysplit Bayer arrangement image 212 from the defect detection andcorrection unit 140, and executes the process of converting the four-waysplit Bayer arrangement into the normal Bayer arrangement, in otherwords, the Bayer arrangement image illustrated in FIG. 1(1). Thearrangement conversion unit 170 generates a Bayer arrangement image 213by the arrangement conversion, and outputs the Bayer arrangement image213 to the camera signal processing unit 180.

The camera signal processing unit 180 inputs the Bayer arrangement image213 output from the arrangement conversion unit 170, and executes thedemosaicing process of setting all RGB colors at the RGB pixel positionsof the Bayer arrangement image. Furthermore, general signal processingin a camera, such as white balance (WB) adjustment and gamma correction,is executed. The output image 300 is generated and output.

The control unit 135 outputs a control signal to each unit in accordancewith a program stored in, for example, the unillustrated memory, andperforms the control of various processes.

[3. Configuration and Process of the Defect Detection and CorrectionUnit]

Next, the configuration and process of the defect detection andcorrection unit 140 are described with reference to FIG. 8.

The defect detection and correction unit 140 includes a local areaextraction unit 141, a defective pixel detection unit 150, and adefective pixel correction unit 160, as illustrated in FIG. 8.

The four-way split Bayer arrangement image 211 output by the imagingdevice 120 is input into the local area extraction unit 141 of thedefective pixel detection and correction unit 140.

The local area extraction unit 141 selects and extracts the pixel valuesof a local area used by the downstream defective pixel detection unit150. The local area is an area corresponding to a pixel cluster as oneunit where the same subject light is diffused and enters. For example,in a case of the configuration illustrated in FIG. 4, pixel areas eachincluding 2×2 pixels are sequentially extracted as the local areas.

Moreover, if the same subject light is diffused and enters in clustersof 4×4 pixels illustrated in FIG. 5(b), then the 4×4-pixel areas areextracted as the local areas.

Moreover, if the same subject light is diffused and enters in clustersof 8×8 pixels illustrated in FIG. 5(c), then the 8×8-pixel areas areextracted as the local areas.

Information on the pixels of the local area extracted by the local areaextraction unit 141 is output to the defective pixel detection unit 150.

The defective pixel detection unit 150 executes the defective pixeldetection process on a local area basis, in other words, on a pixelcluster basis for the diffusion and entrance of the same subject light.

The specific configuration and process of the defective pixel detectionunit 150 are described in detail below.

Information on the defective pixel detected by the defective pixeldetection unit 150 is output to the defective pixel correction unit 160.

The defective pixel correction unit 160 executes the process ofcorrecting the pixel value of the defective pixel identified based onthe information on the defective pixel detected by the defective pixeldetection unit 150.

The specific configuration and process of the defective pixel correctionunit 160 are described in detail below.

The defective pixel correction unit 160 generates the defect-correctedfour-way split Bayer arrangement image 212 where the pixel value of thedefective pixel has been corrected, and outputs the defect-correctedfour-way split Bayer arrangement image 212 to the arrangement conversionunit 170 in the signal processing unit 130 illustrated in FIG. 7.

[3-1. Details of the Configuration and Process of the Defective PixelDetection Unit]

Next, the specific configuration of the defective pixel detection unit150 in the defect detection and correction unit 140 illustrated in FIG.8 is described with reference to FIG. 9.

FIG. 9 is a diagram illustrating the specific configuration of thedefective pixel detection unit 150 in the defect detection andcorrection unit 140 illustrated in FIG. 8.

As illustrated in FIG. 9, the defective pixel detection unit 150includes a maximum/minimum pixel value detection unit 151, a standarddeviation (std) calculation unit 152, an intermediate pixel valueaverage (T) calculation unit 153, and a defective pixel determinationunit 154.

The maximum/minimum pixel value detection unit 151 inputs local areapixel information 401 extracted by the local area extraction unit 141 ofthe defect detection and correction unit 140 illustrated in FIG. 8.

The local area extraction unit 141 extracts the pixel value informationof a pixel cluster where the same subject light is diffused and entersas the local area pixel information 401, as described above. Forexample, in the case of the configuration illustrated in FIGS. 4 and 4(a), one local area is a pixel area including 2×2 pixels. Pieces of thelocal area pixel information 401 are sequentially generated on the localarea basis, and outputs the local area pixel information 401 to themaximum/minimum pixel value detection unit 151 of the defective pixeldetection unit 150 illustrated in FIG. 9.

The maximum/minimum pixel value detection unit 151 detects the maximumand minimum pixel values of the same color pixels included in the localarea pixel information 401.

In the process example described below, a description is given assumingthe local area to be the same color pixel area including 2×2 pixels. Thelocal area of 2×2=4 pixels includes only four pixels of the same color:R, G, or B.

FIG. 9 illustrates an example of inputting 2×2, four R pixels of R1, R2,R3, and R4, in the four-way split Bayer RGB arrangement, as an exampleof the local area pixel information 401.

The maximum/minimum pixel value detection unit 151 detects a pixelhaving the maximum pixel value and a pixel having the minimum pixelvalue from the four same color pixels of 2×2=4 pixels, and outputs thedetection information, together with the local area pixel information401, to the standard deviation (std) calculation unit 152 and theintermediate pixel average (T) calculation unit 153.

The standard deviation (std) calculation unit 152 calculates thestandard deviation (std) of the pixel values of a plurality of the samecolor pixels included in the local area pixel information 401. Thestandard deviation (std) is calculated in accordance with (Equation 1)illustrated below.

[Mathematical Formula 1]

$\begin{matrix}{{{ave} = {\frac{1}{n}{\sum\limits_{i}{Ci}}}}{{std} = \sqrt{\frac{1}{n}{\sum\limits_{i}{{{Ci} - {ave}}}^{2}}}}} & ( {{Equation}\mspace{14mu} 1} )\end{matrix}$

In the above (Equation 1),

ave: the pixel value average of the same color pixels in the local area,

n: the number of the same color pixels in the local area,

i: an index corresponding to each pixel of the same color pixels in thelocal area) i=1, 2, 3, . . . n,

Ci: the pixel value of a pixel, to which an index i has been set, of thesame color pixels (C=R, G, or B) in the local area, and

std: the standard deviation of the pixel values of the same color pixelsin the local area.

For example, if the local area includes four pixels, R1 to R4, asillustrated in FIG. 9, the following values are applied:

n=4,

i=1 to 4, and

Ci=the pixel value of each of R1 to R4.

The value of the standard deviation (std) calculated by the standarddeviation (std) calculation unit 152 in accordance with the above(Equation 1) is input into the defective pixel determination unit 154.

The intermediate pixel average (aye) calculation unit 153 calculates theaverage of the pixel values of pixels having intermediate pixel valuesexcept the maximum and minimum pixel values among the pixel values ofthe plurality of the same color pixels included in the local area pixelinformation 401, in other words, an intermediate pixel average (T). Theintermediate pixel average (T) is calculated in accordance with thefollowing (Equation 2).

[Mathematical Formula 2]

$\begin{matrix}{T = {\frac{1}{n - 2}( {{2{nd}\; {MAX}} + \ldots + {2{nd}\; {Min}}} )}} & ( {{Equation}\mspace{14mu} 2} )\end{matrix}$

In the above (Equation 2),

n: the number of the same color pixels in the local area,

2ndMax: the second maximum pixel value of the same color pixels in thelocal area,

2ndMin: the second minimum pixel value of the same color pixels in thelocal area, and

T: the intermediate pixel average.

For example, if the local area includes four pixels, R1 to R4, asillustrated in FIG. 9, the following values are applied:

n=4,

2ndMax: the pixel value having the second maximum value of R1 to R4, and

2ndMin=the pixel value having the second minimum value of R1 to R4.

The value of the intermediate pixel average (T) calculated by theintermediate pixel average (T) calculation unit 153 in accordance withthe above (Equation 2) is input into the defective pixel determinationunit 154.

The defective pixel determination unit 154 inputs the following values:

(1) the value of the standard deviation (std) calculated by the standarddeviation (std) calculation unit 152 in accordance with the above(Equation 1), and

(2) the value of the intermediate pixel average (T) calculated by theintermediate pixel average (T) calculation unit 153 in accordance withthe above (Equation 2).

The defective pixel determination unit 154 inputs these values, anddetermines whether the pixels of interest selected sequentially from thesame color pixels included in the local area are a defective pixel.

The determination process is performed in accordance with (Equation 3)illustrated below.

[Mathematical Formula 3]

if(std>TH1∩∥T−Ci∥>TH2)  (Equation 3)

In the above (Equation 3),

std: the value of the standard deviation calculated in accordance withthe above (Equation 1),

T: the intermediate pixel average calculated in accordance with theabove (Equation 2),

TH1: the preset threshold value,

TH2: the preset threshold value, and

Ci: the pixel value of a pixel of interest selected sequentially fromthe local area.

If (Equation 3) illustrated above holds for a pixel of interest (Ci)selected sequentially from the local area, the pixel of interest (Ci) isdetermined to be a defective pixel.

On the other hand, if (Equation 3) illustrated above does not hold for apixel of interest (Ci), the pixel of interest (Ci) is determined not tobe a defective pixel.

The above (Equation 3) is an equation for determining a pixel ofinterest (Ci) to be a defective pixel if the following two conditionsare satisfied for the pixel of interest (Ci).

(First Condition) that the standard deviation (std) calculated inaccordance with the above-described (Equation 1), using pixels of thesame color as the pixel of interest (Ci) in the local area including thepixel of interest, is larger than the preset threshold value (TH1).

(Second Condition) that the difference absolute value between theintermediate pixel average (T) calculated in accordance with theabove-described (Equation 2), using pixels of the same color as thepixel of interest (Ci) in the local area including the pixel ofinterest, and the pixel value (Ci) of the pixel of interest is largerthan the preset threshold value (TH2).

In other words, if variations in the pixel values of the same colorpixels included in the local area are large, and

if the difference between the average of the intermediate pixel valuesexcept the maximum and minimum pixel values of the same color pixelsincluded in the local area, and the pixel value of the pixel of interestis large, the pixel of interest is determined to be a defective pixel.

The defective pixel determination unit 154 determines whether or not thepixel of interest Ci selected sequentially from the local area satisfiesthe above (Equation 3). If so, the defective pixel determination unit154 determines the pixel of interest to be a defective pixel and, ifnot, determines the pixel of interest not to be a defective pixel.

The defective pixel determination unit 154 generates defective pixeldetermination information 402 such as a defect determination flagcorresponding to each pixel of interest, as information indicating thedetermination result, and outputs the defective pixel determinationinformation 402 to the downstream defective pixel correction unit 160.

The defect determination flag is, for example, a flag where [1] is setif the pixel of interest is a defective pixel, and [0] is set if it isnot a defective pixel.

The defective pixel detection unit 150 selects the pixels included inthe local area sequentially as the pixel of interest, and repeatedlyexecutes the process of determining whether or not the selected pixel ofinterest is a defective pixel, switching the pixels of interestsequentially.

As a result, all pixels included in the pixels of interest aredetermined whether or not to be a defective pixel.

The local areas are changed and set sequentially to execute a similarprocess on all the pixels forming the image.

Next, the sequence of the defective pixel detection process executed bythe defective pixel detection unit 150 illustrated in FIG. 9 isdescribed with reference to the flowchart illustrated in FIG. 10.

In order to facilitate understanding, specific local area pixelinformation is described taking an example where pixel information of a2×2-pixel area including the R1 to R4 pixels in the center of thefour-way split Bayer RGB arrangement illustrated on the top left cornerof FIG. 10 is input to perform the process.

Firstly, in Step S101, one pixel of interest (Ci) is selected from thelocal area.

The flowchart illustrated in FIG. 10 is repeatedly executed, selectingthe pixels included in the local area sequentially as the pixel ofinterest.

Here, a description is given assuming that the pixel R1 being one of thepixels R1 to R4 forming the local area has been selected as the pixel ofinterest.

Next, in Step S102, the pixel of interest (Ci) is determined whether ornot to have the maximum or minimum pixel value of the same color pixelsof the local area.

If the pixel of interest (Ci=R1) has the maximum or minimum pixel valueof the same color pixels in the local area (the determination of StepS102=Yes), the processing proceeds to Step S103.

On the other hand, if the pixel of interest does not have the maximum orminimum pixel value of the same color pixels in the local area (thedetermination of Step S102=No), the pixel of interest (Ci=R1) isdetermined not to be a defective pixel. The processing then ends.

If the pixel of interest (Ci) is the pixel R1, the followingdetermination process is performed.

The pixel of interest R1 is determined whether or not to be a pixelhaving the maximum pixel value that is a pixel value larger than thepixel values of the other pixels R2, R3, and R4, or whether or not to bea pixel having the minimum pixel value that is a value smaller than thepixel values of the other pixels R2, R3, and R4.

If the pixel of interest is determined to have the maximum or minimumpixel value of the same color pixels in the local area in thedetermination process of Step S102, the processing proceeds to StepS103.

In Step S103, the standard deviation (std) of the pixel values of theplurality of same color pixels included in the local area is calculated.

The process is the process executed by the standard deviation (std)calculation unit 152 described above with reference to FIG. 9.

The standard deviation (std) of the pixel values of the plurality of thesame color pixels included in the local area is calculated in accordancewith the above-described (Equation 1).

For example, if the pixels included in the local area are R1 to R4, thestandard deviation (std) is calculated, setting the pixel values of thefour pixels in Ci of the above-mentioned (Equation 1).

Next, in Step S04, the average of the pixel values of pixels havingintermediate pixel values except the maximum and minimum pixel values ofthe pixel values of the plurality of the same color pixels included inthe local area, in other words, the intermediate pixel average (T), iscalculated.

The process is the process executed by the intermediate pixel average(ave) calculation unit 153 described above with reference to FIG. 9.

The average of the pixel values of the pixels having intermediate pixelvalues except the maximum and minimum pixel values of the pixel valuesof the plurality of the same color pixel values included in the localarea, in other words, the intermediate pixel average (T), is calculatedin accordance with (Equation 2) described above.

For example, if the pixels included in the local area are R1 to R4, theaverage of two pixel values having the intermediate pixel values exceptthe maximum and minimum pixel values among the pixel values of the fourpixels is calculated as the intermediate pixel average (T).

Next, in Step S105, the process of determining whether or not the pixelof interest (Ci) is a defective pixel is executed.

The process is the process executed by the defective pixel determinationunit 154 described above with reference to FIG. 9.

As described above, the defective pixel determination unit 154determines whether or not the pixel of interest (Ci) is a defectivepixel based on the following values:

-   -   (1) the value of the standard deviation (std) calculated by the        standard deviation (std) calculation unit 152 in accordance with        the above (Equation 1), and    -   (2) the value of the intermediate pixel average (T) calculated        by the intermediate pixel average (T) calculation unit 153 in        accordance with the above (Equation 2).

The defective pixel determination unit 154 inputs these values, anddetermines whether the pixels of interest (Ci) selected sequentiallyfrom the same color pixels included in the local area are a defectivepixel.

The determination process is performed in accordance with (Equation 3)illustrated above.

If (Equation 3) holds for a pixel of interest, the pixel of interest isdetermined to be a defective pixel.

On the other hand, if (Equation 3) does not hold for a pixel ofinterest, the pixel of interest is determined not to be a defectivepixel.

If (Equation 3) holds for a pixel of interest, and the pixel of interestis determined to be a defective pixel, the processing proceeds to StepS106, and the value of the defect detection flag being the flagcorresponding to the pixel of interest is set to [1].

On the other hand, if (Equation 3) does not hold for a pixel ofinterest, and the pixel of interest is determined not to be a defectivepixel, the processing is ended. In this case, the value of the defectdetection flag being the flag corresponding to the pixel of interestremains to be set to the initial value [0].

The pixels included in the local area are selected sequentially as thepixel of interest, and the flow illustrated in FIG. 10 is executedsequentially and repeatedly on the selected pixel-of-interest basis.

As a result, all pixels included in the pixels of interest aredetermined whether or not to be a defective pixel.

The local areas are changed and set sequentially to execute a similarprocess on all the pixels forming the image.

[3-2. Details of the Configuration and Process of the Defective PixelCorrection Unit]

Next, the configuration and process of the defective pixel correctionunit 160 in the defect detection and correction unit 140 illustrated inFIG. 8 is described in detail with reference to FIG. 11.

The defective pixel correction unit 160 performs the process ofcorrecting the pixel value of the pixel determined to be a defectivepixel by the upstream defective pixel detection unit 150.

The upstream defective pixel detection unit 150 inputs the defectivepixel determination information (the defect determination flagcorresponding to the pixel) 402, together with the pixel information ofthe local area. The pixel determined to be a defective pixel isidentified based on the flag to correct the pixel value of the defectivepixel.

The intermediate pixel detection unit 161 acquires the following pixelvalues of pixels of the same color as the defective pixel, from thelocal area including the defective pixel:

(a) the pixel value of a pixel having the next pixel value after themaximum pixel value, in other words, the second maximum pixel value, inthe local area (2ndMax), and

(b) the pixel value of a pixel having the next pixel value after theminimum pixel value, in other words, the second minimum pixel value, inthe local area (2ndMin).

The intermediate pixel detection unit 161 acquires these pieces of thepixel value information and outputs them to the pixel value correctionunit 162.

Firstly, the pixel value correction unit 162 determines whether a pixelof interest (Ci) determined to be a defective pixel, which is acorrection target, is a pixel having the maximum or minimum value amongthe same color pixels in a local area to which the pixel of interestbelongs.

If the pixel of interest is the maximum value pixel in the local area towhich the pixel of interest belongs, a pixel value correction is madewhere the pixel value of the pixel of interest is set to the pixel value(2ndMax) input from the intermediate pixel detection unit 161.

In other words, the pixel value of the pixel of interest is corrected tothe pixel value of a pixel having the next pixel value after the maximumpixel value, in other words, the second maximum pixel value, of the samecolor in the local area (2ndMax).

On the other hand, if the pixel of interest is the minimum value pixelin the local area to which the pixel of interest belongs, a pixel valuecorrection is made where the pixel value of the pixel of interest is setto the pixel value (2ndMin) input from the intermediate pixel detectionunit 161.

In other words, the pixel value of the pixel of interest is corrected tothe pixel value of a pixel having the next pixel value after the minimumpixel value, in other words, the second minimum pixel value, of the samecolor in the local area (2ndMin).

The defective pixel correction unit 160 corrects defective pixels inlocal areas sequentially.

Moreover, the local areas are changed and set sequentially. A similarcorrection process is executed on all defective pixels forming theimage.

With the process, all the defective pixels of the process target imageare corrected. The defect-corrected four-way split Bayer arrangementimage 212 illustrated in FIG. 11 is generated and output to thedownstream arrangement conversion unit 170 (refer to FIG. 7).

Next, the detailed sequence of the defective pixel correction processexecuted by the defective pixel correction unit 160 is described withreference to the flowchart illustrated in FIG. 12.

Firstly, in Step S201, one pixel of interest (Ci) is input.

The flow illustrated in FIG. 12 is executed repeatedly on all the pixelsthe pixels in the process image, setting the pixels sequentially as thepixel of interest.

Next, in Step S202, it is determined whether or not the setting of thedefect detection flag of the pixel of interest is set to the valueindicating to be a defective pixel, [1].

If the defect detection flag=1, the processing proceeds to Step S203.

On the other hand, if the defect detection flag=0, in other words,indicates not to be a defective pixel, the processing is ended withoutmaking a correction.

If the value of the defect detection flag of the pixel of interest (Ci)is [1], and the pixel of interest is a defective pixel, the processingproceeds to Step S203.

In Step S203, the pixel values of the second maximum pixel value(2ndMax) and the second minimum pixel value (2ndMin) among the pixels ofthe same color as the pixel of interest are acquired from a local areato which the pixel of interest belongs.

Next, in Step S204, it is determined whether the pixel of interest (Ci)is a pixel having the maximum or minimum pixel value among the samecolor pixels in the local area to which the pixel of interest belongs.

If it is determined that the pixel of interest (Ci) is a pixel havingthe maximum pixel value among the same color pixels in the local area towhich the pixel of interest belongs, the processing proceeds to StepS205. In Step S205, the pixel value of the pixel of interest is set tothe second maximum pixel value (2ndMax) among the pixels of same coloras the pixel of interest in the local area to which the pixel ofinterest belongs. In other words, the pixel value (Ci) of the pixel ofinterest is set as follows:

Ci=2ndMax

On the other hand, if it is determined in Step S204 that the pixel ofinterest (Ci) is a pixel having the minimum pixel value among the samecolor pixels in the local area to which the pixel of interest belongs,the processing proceeds to Step S206. In Step S206, the pixel value ofthe pixel of interest is set to the second minimum pixel value (2ndMin)among the pixels of the same color as the pixel of interest in the localarea to which the pixel of interest belongs. In other words, the pixelvalue (Ci) of the pixel of interest is set as follows:

Ci=2ndMin

The defective pixel correction unit 160 sets the pixels in the processtarget image sequentially as the pixel of interest, and repeatedlyexecutes the process in accordance with the flow illustrated in FIG. 12.

With the process, all the defective pixels of the process target imageare corrected. The defect-corrected four-way split Bayer arrangementimage 212 illustrated in FIG. 11 is generated and output to thedownstream arrangement conversion unit 170 (refer to FIG. 7).

[4. Example Using a Low-Resolution Lens for the Imaging Lens]

Next, an example where a low-resolution lens is used for the imaginglens is described as a second example of the imaging apparatus of thepresent disclosure.

As described above with reference to FIG. 4 and the like, the imagingapparatus of the present disclosure has a configuration where the samesubject light is diffused over and enters a plurality of pixels of theimaging device.

In other words, it is configured to acquire an image with a loweroptical resolving power than a pixel resolving power corresponding tothe pixel density of the imaging device.

In order to acquire such an image, the above-mentioned first example isconfigured in such a manner as to place the microlens 124 between theimaging lens 110 and the photoelectric conversion element 121, anddiffuse incident light from the imaging lens 110 as described withreference to FIG. 4 and the like.

In this manner, it is configured in such a manner as that the samesubject light is blurred on purpose and applied to a plurality of pixelsof the imaging device and accordingly light of uniform intensity isapplied to the plurality of pixels. The pixel values of the same colorpixels in the local area are compared to enable the highly accuratedetection of a defective pixel.

A configuration other than the configuration using the microlensillustrated in FIG. 4 is also feasible as the configuration to apply thesame subject light to a plurality of pixels of the imaging device.

For example, as illustrated in FIG. 13, a configuration using alow-resolution imaging lens 510 enables the application of the samesubject light to a plurality of pixels of the imaging device.

In the configuration illustrated in FIG. 13, the low-resolution imaginglens 510 is a lens with lower resolution than the pixel pitch of theimaging device. In terms of incident light entering via thelow-resolution imaging lens 510, it is set in such a manner as that thesame subject light is applied to a pixel area of a plurality of pixels,specifically, 2×2 pixels as illustrated in FIG. 13 (a), even in thesharpest focus position.

In this manner, a lens with lower resolution than the pixel pitch isapplied. Accordingly, the light from a subject in focus can be diffusedover and applied to a plurality of pixels.

The configuration of an imaging device 520 illustrated in FIG. 13corresponds to a configuration where the microlenses 124 are omittedfrom the imaging device 120 described above with reference to FIG. 4. Inother words, the imaging device 520 has a configuration where aphotoelectric conversion element 521, a color filter 522, and an onchiplens 523 are stacked.

The pixel arrangement of the imaging device 520 is the four-way splitBayer arrangement described with reference to FIG. 1(2) as in the firstexample.

[5. Example Using an Optical Low-Pass Filter]

Next, an example where an optical low-pass filter is used is describedas a third example of the imaging apparatus of the present disclosure.

In the first example, the configuration is achieved where the samesubject light is diffused over and enters a plurality of pixels of theimaging device by use of the microlens 124 as illustrated in FIG. 4.

Moreover, in the above second example, the configuration is achievedwhere the same subject light is diffused over and enters a plurality ofpixels of the imaging device by use of the low-resolution imaging lens510 as illustrated in FIG. 13.

In addition, in order to achieve a configuration where the same subjectlight is diffused over and enters a plurality of pixels of the imagingdevice, it may be configured to place an optical low-pass filter 624that selectively transmits only a low frequency light between an imaginglens 610 and a photoelectric conversion element 621 as illustrated inFIG. 14.

In the configuration illustrated in FIG. 14, the optical low-pass filter624 is an optical low-pass filter having a property that does nottransmit a predetermined high frequency light from incident light viathe imaging lens 610.

The light transmitted through the optical low-pass filter 624 plays arole in spreading, that is, blurring, luminous flux corresponding to thesame subject light as illustrated in FIG. 14. As a result, the samesubject light is diffused over and enters a plurality of pixels of theimaging device as in the first and second examples described above.Specifically, it is set in such a manner as that the same subject lightis applied to a pixel area of 2×2 pixels as illustrated in FIG. 14(a).

The configuration of an imaging device 620 illustrated in FIG. 14corresponds to a configuration where the microlenses 124 in the imagingdevice 120 described above with reference to FIG. 4 are replaced withthe optical low-pass filter 624. The other configurations are similar tothose illustrated in FIG. 4. The configuration of the imaging device 620is a configuration where a photoelectric conversion element 621, a colorfilter 622, and an onchip lens 623 are stacked and the optical low-passfilter 624 is placed on the imaging lens side.

The arrangement of the pixels of the imaging device 520 is the four-waysplit Bayer arrangement described with reference to FIG. 1(2) as in thefirst example.

[6. Variations in the Pixel Arrangement]

In the first example described above, a description has been given ofthe configuration where the four-way split Bayer RGB arrangementillustrated in FIG. 1(2) is used as the arrangement of the pixels of theimaging device.

The processes of the present disclosure are not limited to the four-waysplit Bayer RGB arrangement illustrated in FIG. 1(2) and can also beapplied to imaging devices of other various pixel arrangements.

It can also be applied to an imaging device having such a four-way splitWRGB arrangement configuration as illustrated in FIG. 15(B).

The arrangement illustrated in FIG. 15(B) is an imaging device havingthe four-way split WRGB arrangement configuration including a White(transparent) pixel that transmits almost all the visible wavelengthlights. Also if an imaging device having such a pixel arrangement isused, the defective pixel detection and correction processes inaccordance with the above-mentioned examples are feasible.

If the WRGB arrangement illustrated in FIG. 15(B) is used, thearrangement conversion unit 170 of the signal processing unit 130illustrated in FIG. 7 needs to execute the process of converting thefour-way split WRGB arrangement into the Bayer arrangement.

[7. Example in which the Arrangement Conversion Unit is Replaced withthe Demosaicing Processing Unit]

The signal processing unit 130 described above with reference to FIG. 7inputs an image where the pixel values after the defect correctionoutput from the defect detection and correction unit 140 are set, inother words. The defect-corrected four-way split Bayer arrangement image212 illustrated in FIG. 7, is input into the arrangement conversion unit170. The arrangement conversion unit 170 is configured to convert thepixel arrangement of the defect-corrected four-way split Bayerarrangement image 212 being the input image, generate the Bayerarrangement image 213, and input the Bayer arrangement image 213 intothe camera signal processing unit 180.

In other words, in the above-mentioned first example, the camera signalprocessing unit 180 is configured to execute the demosaicing process, inother words, the demosaicing process of setting previous RGB pixelvalues for each pixel based on the Bayer arrangement image 213 whereonly a pixel value of R, G, or B has been set for each pixel.

In contrast, the signal processing unit 130 illustrated in FIG. 16includes a demosaicing processing unit 701 instead of the arrangementconversion unit 170 illustrated in FIG. 7.

The demosaicing processing unit 701 inputs the defect-corrected four-waysplit Bayer arrangement image 212 from the defect detection andcorrection unit 140 into the arrangement conversion unit 170, andexecutes the demosaicing process of setting all RGB pixels for eachpixel of the converted Bayer arrangement image based on the input imagein conjunction with the conversion into the Bayer arrangement. An RGBimage 711 generated by the demosaicing process is output to the camerasignal processing unit 180.

The camera signal processing unit 180 executes other general camerasignal processing, for example, processes such as white balanceadjustment and gamma correction, without executing the demosaicingprocess, and generates an output image 300.

[8. Example Including a Specific Point-of-View Image Generation Unit]

Next, an example including a specific point-of-view image generationunit 801 in the signal processing unit 130 is described with referenceto FIG. 17.

The first example described above has a configuration where themicrolenses 124 are placed on the front side of the imaging device 120as described with reference to FIG. 4.

The configuration where the placement of the microlens 124 allows thesubject light to be diffused over and applied to a plurality of pixelsis as described above.

As another action of the microlens 124, there is action that an imagefrom a different point-of-view, or an image with a different focallength, depending on the area of the photoelectric conversion element ofthe imaging device, can be acquired individually.

For example, an imaging method called “Light Field Photography” isdisclosed in the document [Ren. Ng, and 7 others, “Light FieldPhotography with a Hand-Held Plenoptic Camera”, Stanford Tech ReportCTSR 2005-02].

The method discloses a configuration where a microlens is placed at thefront of the imaging device, and incident light is diffused via themicrolens and applied to photoelectric conversion units as in the firstexample described above. This configuration enables the individualacquisition of an image from a different point-of-view, or an image witha different focal length, depending on the area of the photoelectricconversion element of the imaging device.

In accordance with “Light Field Photography” described in the abovedocument, the configuration to individually acquire an image from adifferent point-of-view, or an image with a different focal length,depending on the area of the photoelectric conversion element of theimaging device is described with reference to FIG. 18.

FIG. 18 illustrates a subject 900 as an image capture target, an imaginglens 921, microlenses 922, and photoelectric conversion elements 923.

Light emitted from a specific position from the head to the toe of thesubject 900 passes through the imaging lens 921 as a main lens, and iscollected onto one of the microlenses 922. For example, the ray of lightemitted from the head of the subject 900 in the figure is collected ontoa microlens 931 in the lower part of the imaging device as in thefigure, and the ray of light emitted from the toe of the subject 900 issimilarly condensed onto a microlens 932 in the upper part.

The lights having passed through the microlenses 922 are applied topixels of the photoelectric conversion elements 923 in the correspondingpatches.

In this manner, it is possible to record the generation position of theray of light emitted from the subject in the microlens 922, and recordthe direction of the ray of light in the imaging device. In this manner,the ray of light recorded in each pixel of the photoelectric conversionelement 923 is selectively used. Accordingly, a specific focal pointimage and a specific point-of-view image can be generated.

For example, an image is restored by collecting only the rays of lightdrawn by solid lines (A) of FIG. 18. Accordingly, an image that the raysof light pass the lower end of the imaging lens 921 can be generated.Moreover, if the rays of light drawn by dotted lines (B) of FIG. 18 arecollected, an image that the rays of light have passed through thecenter of the imaging lens 921 can be obtained. Furthermore, if the raysof light drawn by dot and dash lines (C) of FIG. 18 are collected, animage that the rays of light have passed through the top of the imaginglens 921 can be obtained.

An image having a plurality of points of sight can be generated in oneimage capture in this manner. Alternatively, an image having a specificblur can be generated by synthesizing the rays of light obtained fromdifferent points of sight.

The images captured in such a method are used. The information on thedirection and position of the ray of light of the subject correspondingto each pixel of the imaging device is referred to and the images areappropriately combined and reconstructed. Therefore, it becomes possibleto generate an image of different points of sight using one capturedimage. Alternatively, a plurality of images having different focalpoints can be generated simultaneously.

The specific point-of-view image generation unit 801 in the signalprocessing unit 130 illustrated in FIG. 17 executes these processes,generates an image of a plurality of different points of sight, or aplurality of images having different focal points as a specificpoint-of-view image 811, and outputs the specific point-of-view image811 to the arrangement conversion unit 170.

The specific point-of-view image 811 illustrated in the figure includesone or a plurality of specific point-of-view images.

Here, the specific point-of-view is assumed to include both thepoint-of-view as an observation position and the position of the subjectat a specific focal length.

In this manner, the specific point-of-view image generation unit 801 isset. Accordingly, different specific point-of-view images can begenerated simultaneously with the defect correction.

[9. Summary of the Configuration of the Present Disclosure]

Up to this point the examples of the present disclosure have beendescribed in detail with reference to the specific examples. However, itis obvious that those skilled in the art can make modifications to andsubstitutions of the examples within the range that does not deviatefrom the gist of the present disclosure. In other words, the presentinvention has been disclosed by way of illustration. The presentinvention should not be interpreted restrictively. In order to judge thegist of the present disclosure, the claims should be taken intoconsideration.

The technology disclosed in the description can take the followingconfigurations:

(1) An imaging apparatus including:

an imaging device; and

a signal processing unit for analyzing an output signal from the imagingdevice and detecting a defective pixel included in the imaging device,wherein

the imaging device inputs the same subject light on a local area basisincluding a plurality of pixels of the imaging device, acquires an imagesignal lower than a pixel resolving power corresponding to the pixeldensity of the imaging device, and outputs the image signal to thesignal processing unit, and

the signal processing unit compares pixel values of the same colorpixels included in the local area on a local area basis including acluster of the plurality of pixels of the imaging device, and detects adefective pixel based on the comparison result.

(2) The imaging apparatus according to (1), wherein the signalprocessing unit executes a process of determining a pixel of interest tobe a defective pixel when variations in pixel values of the same colorpixels included in the local area is large, and when a differencebetween an average of intermediate pixel values except maximum andminimum pixel values of the same color pixels included in the localarea, and a pixel value of the pixel of interest is large.(3) The imaging apparatus according to (1) or (2), wherein the signalprocessing unit executes a process of determining a pixel of interest tobe a defective pixel when a standard deviation of pixel values of thesame color pixels included in the local area is larger than a presetthreshold value (TH1), and when a difference absolute value between anaverage of intermediate pixel values except maximum and minimum pixelvalues of the same color pixels included in the local area, and a pixelvalue of the pixel of interest is larger than a preset threshold value(TH2).(4) The imaging apparatus according to any of (1) to (3), wherein theimaging device includes a photoelectric conversion unit having pixelsarranged in a two-dimensional array form, and a microlens placed on animaging lens side being on a front side of the photoelectric conversionunit, and has a configuration where incident light via an imaging lensis diffused via the microlens, and the same subject light is received onthe local area basis including the plurality of pixels of the imagingdevice.(5) The imaging apparatus according to (4), wherein the signalprocessing unit has a configuration where an image on a specific pixelarea basis is reconstructed from an image acquired by the imaging deviceincluding the microlens and accordingly a specific point-of-view imageis generated.(6) The imaging apparatus according to any of (1) to (5), wherein theimaging device has a pixel arrangement where each pixel of a Bayerarrangement including RGB pixels or an arrangement including RGBW pixelsis split into four, 2×2 pixels, of the same color.(7) The imaging apparatus according to (6), wherein the imaging deviceinputs the same subject light on a local area basis including 2×2 samecolor pixels of the imaging device, acquires an image signal lower thana pixel resolving power corresponding to the pixel density of theimaging device, and outputs the image signal to the signal processingunit.(8) The imaging apparatus according to (6), wherein the imaging deviceinputs the same subject light on a local area basis including 4×4 or 8×8pixels of the imaging device, acquires an image signal lower than apixel resolving power corresponding to the pixel density of the imagingdevice, and outputs the image signal to the signal processing unit.(9) The imaging apparatus according to any of (1) to (8), wherein theimaging device lets in subject light via a low-resolution imaging lenswith a low optical resolving power for forming an optical image with aresolving power lower than a pixel resolving power corresponding to thepixel density of the imaging device and accordingly acquires an imagesignal lower than the pixel resolving power corresponding to the pixeldensity of the imaging device, and outputs the image signal to thesignal processing unit.(10) The imaging apparatus according to any of (1) to (9), wherein theimaging device includes a photoelectric conversion unit having pixelsarranged in a two-dimensional array form, and an optical low-pass filterplaced on an imaging lens side being on a front side of thephotoelectric conversion unit, and has a configuration where incidentlight via an imaging lens is diffused via the optical low-pass filter,and the same subject light is received on the local area basis includingthe plurality of pixels of the imaging device.(11) The imaging apparatus according to any of (1) to (10), wherein thesignal processing unit includes a defective pixel correction unit forexecuting a pixel value correction on a defective pixel, and thedefective pixel correction unit calculates a corrected pixel value ofthe defective pixel taking, as reference pixels, intermediate pixelvalues except maximum and minimum pixel values of the pixels of the samecolor as the defective pixel included in the same local area as thedefective pixel.(12) The imaging apparatus according to (11), wherein the defectivepixel correction unit sets a maximum pixel value of the intermediatepixel values as the corrected pixel value of the defective pixel whenthe defective pixel has the maximum pixel value among the same colorpixels in the local area including the defective pixel, and sets aminimum pixel value of the intermediate pixel values as the correctedpixel value of the defective pixel when the defective pixel has theminimum pixel value among the same color pixels in the local areaincluding the defective pixel.(13) An image processing method executed in an imaging apparatus,executing:

a step of, in an imaging device, inputting the same subject light on alocal area basis including a plurality of pixels of the imaging device,acquiring an image signal lower than a pixel resolving powercorresponding to the pixel density of the imaging device, and outputtingthe image signal to the signal processing unit, and

a signal processing step of, in the signal processing unit, analyzing anoutput signal from the imaging device, and detecting a defective pixelincluded in the imaging device, wherein

in the signal processing step, the signal processing unit compares pixelvalues of the same color pixels included in the local area on a localarea basis including a cluster of the plurality of pixels of the imagingdevice, and detects the defective pixel based on the comparison result.

(14) A program to cause an imaging apparatus to execute imageprocessing, including:

a step of causing an imaging device to input the same subject light on alocal area basis including a plurality of pixels of the imaging device,acquire an image signal lower than a pixel resolving power correspondingto the pixel density of the imaging device, and output the image signalto the signal processing unit, and

a signal processing step of causing the signal processing unit toanalyze an output signal from the imaging device, and detect a defectivepixel included in the imaging device, wherein

the signal processing step includes comparing pixel values of the samecolor pixels included in the local area on a local area basis includinga cluster of the plurality of pixels of the imaging device, anddetecting the defective pixel based on the comparison result.

Moreover, a series of processes described in the description can beexecuted by hardware, software, or a configuration of their combination.In a case of executing the processes by software, a program in which theprocess sequence is recorded can be installed in memory in a computerintegrated in dedicated hardware, and executed, or installed in ageneral purpose computer capable of executing various processes, andexecuted. For example, the program can be prerecorded in a recordingmedium. In addition to installation from a recording medium to acomputer, the program can be installed in a built-in recording mediumsuch as a hard disk by being received via a network such as a LAN (LocalArea Network) or the Internet.

Various processes described in the description are not only executed inchronological order in accordance with the description, but can beexecuted in parallel or individually in accordance with the processingcapacity of a device that executes the processes, or as needed.Moreover, the system in the description is a logically assembledconfiguration of a plurality of devices, and is not limited to onehaving devices of configurations in the same housing.

INDUSTRIAL APPLICABILITY

As described above, according to the configuration of one example of thepresent disclosure, a defective pixel of an imaging device is detected.An output pixel value of the defective pixel is corrected to generate anoutput image.

Specifically, an imaging apparatus includes an imaging device, and asignal processing unit that analyzes an output signal from the imagingdevice and detects a defective pixel. The imaging device receivesincident light via, for example, a microlens placed in front of a pixel,inputs the same subject light on a local area basis including aplurality of pixels of the imaging device, and acquires an image signallower than a pixel resolving power corresponding to the pixel density ofthe imaging device. The signal processing unit compares the pixel valuesof the same color pixels included in the local area on a local areabasis including a cluster of a plurality of pixels of the imagingdevice, detects a defective pixel based on the comparison result, andcorrects and outputs a pixel value of a pixel determined to be adefective pixel.

These processes enable highly accurate detection of a defective pixelexisting in the imaging device. Accordingly, it is possible to generatea high quality output image where the output pixel value of thedefective pixel has been corrected.

REFERENCE SIGNS LIST

-   110 Imaging lens-   120 Imaging device-   121 Photoelectrical conversion element-   122 Color filter-   123 Onchip lens-   124 Microlens-   130 Signal processing unit-   135 Control unit-   140 Defect detection and correction unit-   141 Local area extraction unit-   150 Defective pixel detection unit-   151 Maximum/minimum pixel value detection unit-   152 Standard deviation (std) calculation unit-   153 Intermediate pixel average (T) calculation unit-   154 Defective pixel determination unit-   160 Defective pixel correction unit-   161 Intermediate pixel detection unit-   162 Pixel value correction unit-   510 Low-resolution imaging lens-   520 Imaging device-   521 Photoelectric conversion element-   522 Color filter-   523 Onchip lens-   610 Imaging lens-   620 Imaging device-   621 Photoelectric conversion element-   622 Color filter-   623 Onchip lens-   624 Optical low-pass filter-   701 Demosaicing processing unit-   801 Specific point-of-view image generation unit-   900 Subject-   921 Imaging lens-   922 Microlens-   923 Photoelectric conversion element

1. An imaging apparatus, comprising: an image sensor; and signalprocessing circuitry configured to analyze an output signal from theimage sensor and detect a defective pixel included in the image sensor,wherein: the image sensor is configured to input the same subject lighton a local area basis via a low-resolution imaging lens with a lowoptical resolving power for forming an optical image with a resolvingpower lower than a pixel resolving power corresponding to the pixeldensity of the image sensor and accordingly acquire an image signallower than the pixel resolving power corresponding to the pixel densityof the image sensor, and output the image signal to the signalprocessing circuitry, and the signal processing circuitry is configuredto compare pixel values of the same color pixels included in the localarea on a local area basis including a cluster of the plurality ofpixels of the image sensor, and detect a defective pixel based on thecomparison result.
 2. The imaging apparatus according to claim 1,wherein the signal processing circuitry is further configured to executea process of determining a pixel of interest to be a defective pixelwhen variations in pixel values of the same color pixels included in thelocal area is large, and when a difference between an average ofintermediate pixel values except maximum and minimum pixel values of thesame color pixels included in the local area, and a pixel value of thepixel of interest is large.
 3. The imaging apparatus according to claim1, wherein the signal processing circuitry is further configured toexecute a process of determining a pixel of interest to be a defectivepixel when a standard deviation of pixel values of the same color pixelsincluded in the local area is larger than a preset threshold value(TH1), and when a difference absolute value between an average ofintermediate pixel values except maximum and minimum pixel values of thesame color pixels included in the local area, and a pixel value of thepixel of interest is larger than a preset threshold value (TH2).
 4. Theimaging apparatus according to claim 1, wherein the image sensorincludes a photoelectric conversion unit having pixels arranged in atwo-dimensional array form, and a microlens placed on an imaging lensside being on a front side of the photoelectric conversion unit, and hasa configuration where incident light via an imaging lens is diffused viathe microlens, and the same subject light is received on the local areabasis including the plurality of pixels of the image sensor.
 5. Theimaging apparatus according to claim 4, wherein the signal processingcircuitry has a configuration where an image on a specific pixel areabasis is reconstructed from an image acquired by the image sensorincluding the microlens and accordingly a specific point-of-view imageis generated.
 6. The imaging apparatus according to claim 1, wherein theimage sensor has a pixel arrangement where each pixel of a Bayerarrangement including RGB pixels or an arrangement including RGBW pixelsis split into four, 2×2 pixels, of the same color.
 7. The imagingapparatus according to claim 6, wherein the image sensor is furtherconfigured to input the same subject light on a local area basisincluding 2×2 same color pixels of the image sensor, acquire an imagesignal lower than a pixel resolving power corresponding to the pixeldensity of the image sensor, and output the image signal to the signalprocessing circuitry.
 8. The imaging apparatus according to claim 6,wherein the image sensor is further configured to input the same subjectlight on a local area basis including 4×4 or 8×8 pixels of the imagesensor, acquire an image signal lower than a pixel resolving powercorresponding to the pixel density of the image sensor, and output theimage signal to the signal processing circuitry.
 9. The imagingapparatus according to claim 1, wherein the image sensor includes aphotoelectric conversion unit having pixels arranged in atwo-dimensional array form, and an optical low-pass filter placed on animaging lens side being on a front side of the photoelectric conversionunit, and has a configuration where incident light via an imaging lensis diffused via the optical low-pass filter, and the same subject lightis received on the local area basis including the plurality of pixels ofthe image sensor.
 10. The imaging apparatus according to claim 1,wherein the signal processing circuitry is further configured to executea pixel value correction on a defective pixel, and calculate a correctedpixel value of the defective pixel taking, as reference pixels,intermediate pixel values except maximum and minimum pixel values of thepixels of the same color as the defective pixel included in the samelocal area as the defective pixel.
 11. The imaging apparatus accordingto claim 10, wherein the signal processing circuitry is furtherconfigured to set a maximum pixel value of the intermediate pixel valuesas the corrected pixel value of the defective pixel when the defectivepixel has the maximum pixel value among the same color pixels in thelocal area including the defective pixel, and set a minimum pixel valueof the intermediate pixel values as the corrected pixel value of thedefective pixel when the defective pixel has the minimum pixel valueamong the same color pixels in the local area including the defectivepixel.
 12. An imaging apparatus, comprising: an image sensor; and signalprocessing circuitry configured to analyze an output signal from theimage sensor and detect a defective pixel included in the image sensor,wherein: the image sensor is configured to input the same subject lighton a local area basis including a plurality of pixels of the imagesensor, acquire an image signal lower than a pixel resolving powercorresponding to the pixel density of the image sensor, and output theimage signal to the signal processing circuitry, the signal processingcircuitry is configured to compare pixel values of the same color pixelsincluded in the local area on a local area basis including a cluster ofthe plurality of pixels of the image sensor, and detect a defectivepixel based on the comparison result, and the signal processingcircuitry is further configured to operate as a defective pixelcorrection unit for executing a pixel value correction on a defectivepixel, wherein the defective pixel correction unit calculates acorrected pixel value of the defective pixel taking, as referencepixels, intermediate pixel values except maximum and minimum pixelvalues of the pixels of the same color as the defective pixel includedin the same local area as the defective pixel, sets a maximum pixelvalue of the intermediate pixel values as the corrected pixel value ofthe defective pixel when the defective pixel has the maximum pixel valueamong the same color pixels in the local area including the defectivepixel, and sets a minimum pixel value of the intermediate pixel valuesas the corrected pixel value of the defective pixel when the defectivepixel has the minimum pixel value among the same color pixels in thelocal area including the defective pixel.
 13. The imaging apparatusaccording to claim 12, wherein the signal processing circuitry isfurther configured to execute a process of determining a pixel ofinterest to be a defective pixel when variations in pixel values of thesame color pixels included in the local area is large, and when adifference between an average of intermediate pixel values exceptmaximum and minimum pixel values of the same color pixels included inthe local area, and a pixel value of the pixel of interest is large. 14.The imaging apparatus according to claim 12, wherein the signalprocessing circuitry is further configured to execute a process ofdetermining a pixel of interest to be a defective pixel when a standarddeviation of pixel values of the same color pixels included in the localarea is larger than a preset threshold value (TH1), and when adifference absolute value between an average of intermediate pixelvalues except maximum and minimum pixel values of the same color pixelsincluded in the local area, and a pixel value of the pixel of interestis larger than a preset threshold value (TH2).
 15. The imaging apparatusaccording to claim 12, wherein the image sensor includes a photoelectricconversion unit having pixels arranged in a two-dimensional array form,and a microlens placed on an imaging lens side being on a front side ofthe photoelectric conversion unit, and has a configuration whereincident light via an imaging lens is diffused via the microlens, andthe same subject light is received on the local area basis including theplurality of pixels of the image sensor.
 16. The imaging apparatusaccording to claim 15, wherein the signal processing circuitry has aconfiguration where an image on a specific pixel area basis isreconstructed from an image acquired by the image sensor including themicrolens and accordingly a specific point-of-view image is generated.17. The imaging apparatus according to claim 12, wherein the imagesensor has a pixel arrangement where each pixel of a Bayer arrangementincluding RGB pixels or an arrangement including RGBW pixels is splitinto four, 2×2 pixels, of the same color.
 18. The imaging apparatusaccording to claim 17, wherein the image sensor is further configured toinput the same subject light on a local area basis including 2×2 samecolor pixels of the image sensor, acquire an image signal lower than apixel resolving power corresponding to the pixel density of the imagesensor, and output the image signal to the signal processing circuitry.19. The imaging apparatus according to claim 17, wherein the imagesensor is further configured to input the same subject light on a localarea basis including 4×4 or 8×8 pixels of the image sensor, acquire animage signal lower than a pixel resolving power corresponding to thepixel density of the image sensor, and output the image signal to thesignal processing circuitry.
 20. The imaging apparatus according toclaim 12, wherein the image sensor includes a photoelectric conversionunit having pixels arranged in a two-dimensional array form, and anoptical low-pass filter placed on an imaging lens side being on a frontside of the photoelectric conversion unit, and has a configuration whereincident light via an imaging lens is diffused via the optical low-passfilter, and the same subject light is received on the local area basisincluding the plurality of pixels of the image sensor.